Patterns In Data Research Articles

Carcass traits and morphometry, typification of the Longissimus dorsi muscle and non-carcass components of hair lambs: can biscuit bran completely replace corn? A machine learning approach.

Biscuit bran (BB) is a co-product with worldwide distribution, with Brazil as the second largest cookie producer in the world with 1,157,051 tons. We evaluate the impact of completely replacing corn with BB on the characteristics and morphometry of carcass of purebred and crossbred Morada Nova lambs using machine learning techniques as an auxiliary method. Twenty male lambs from two genetic groups (GG) were used: purebred red-coated Morada Nova (MNR) and crossbred MNR × white-coated Morada Nova (MNF1). Supervised and unsupervised machine learning techniques were used. No interaction (P > 0.05) was observed between diets (D) and genetic groups (GG) and no simple isolated effect was observed for carcass characteristics, qualitative-quantitative typification of the Longissimus dorsi muscle, weight of non-carcass components, weight and yield of commercial cuts and carcass morphometric measurements. The formation of two horizontal clusters was verified: (i) crossed lambs with corn and BB and (ii) purebred lambs fed corn and BB. Vertically, three clusters were formed based on carcass and meat characteristics of native lambs: (i) thermal insulation, body capacity, true yield, and commercial cuts; (ii) choice, performance, physical carcass traits, and palatability; and (iii) yield cuts and non-carcass components. The heatmap also allowed us to observe that pure MN lambs had a greater body capacity when fed BB, while those fed corn showed superiority in commercial cuts, true yields, and non-carcass components. Crossbred lambs, regardless of diet, showed a greater association of physical characteristics of the carcass, performance, palatability, and less noble cuts. Crossbred lambs, regardless of diet, showed a greater association of physical characteristics of the carcass, performance, palatability, and less noble cuts. BB can be considered an alternative energy source in total replacement of corn. Integrating of machine learning techniques is a useful statistical tool for studies with large numbers of variables, especially when it comes to analyzing complex data with multiple effects in the search for data patterns and insights in decision-making on the farm.

Data Validation Utilizing Expert Knowledge and Shape Constraints

Data validation is a primary concern in any data-driven application, as undetected data errors may negatively affect machine learning models and lead to suboptimal decisions. Data quality issues are usually detected manually by experts, which becomes infeasible and uneconomical for large volumes of data. To enable automated data validation, we propose “shape constraint-based data validation”, a novel approach based on machine learning that incorporates expert knowledge in the form of shape constraints. Shape constraints can be used to describe expected (multivariate and nonlinear) patterns in valid data, and enable the detection of invalid data which deviates from these expected patterns. Our approach can be divided into two steps: (1) shape-constrained prediction models are trained on data, and (2) their training error is analyzed to identify invalid data. The training error can be used as an indicator for invalid data because shape-constrained models can fit valid data better than invalid data. We evaluate the approach on a benchmark suite consisting of synthetic datasets, which we have published for benchmarking similar data validation approaches. Additionally, we demonstrate the capabilities of the proposed approach with a real-world dataset consisting of measurements from a friction test bench in an industrial setting. Our approach detects subtle data errors that are difficult to identify even for domain experts.

Visualization as irritation: producing knowledge about medieval courts through uncertainty

Visualizations are ubiquitous in data-driven research, serving as both tools for knowledge production and genuine means of knowledge communication. Despite criticisms targeting the alleged objectivity of visualizations in the digital humanities (DH) and reflections on how they may serve as representations of both scholarly perspective and uncertainty within the data analysis pipeline, there remains a notable scarcity of in-depth theoretical grounding for these assumptions in DH discussions. It is our understanding that only through theoretical foundations such as basic semiotic principles and perspectives on media modality one can fully assess the use and potential of visualizations for innovation in scholarly interpretation. We argue that visualizations have the capacity to “productively irritate” existing scholarly knowledge in a given research field. This does not just mean that visualizations depict patterns in datasets that seem not in line with prior research and thus stimulate deeper examination. Complementarily, “irritation” here consists of visualizations producing uncertainty about their own meaning—yet it is precisely this uncertainty in which the potential for greater insight lies. It stimulates questions about what is depicted and what is not. This turns out to be a valuable resource for scholarly interpretation, and one could argue that visualizing big data is particularly prolific in this sense, because due to their complexity researchers cannot interpret the data without visual representations. However, we argue that “productive irritation” can also happen below the level of big data. We see this potential rooted in the genuinely semiotic and semantic properties of visual media, which studies in multimodality and specifically in the field of Bildlinguistik have carved out: a visualization's holistic overview of data patterns is juxtaposed to its semantic vagueness, which gives way to deep interpretations and multiple perspectives on that data. We elucidate this potential using examples from medieval English legal history. Visualizations of data relating to legal functions and social constellations of various people in court offer surprising insights that can lead to new knowledge through “productive irritation.”

A novel multi-task machine learning classifier for rare disease patterning using cardiac strain imaging data

To provide accurate predictions, current machine learning-based solutions require large, manually labeled training datasets. We implement persistent homology (PH), a topological tool for studying the pattern of data, to analyze echocardiography-based strain data and differentiate between rare diseases like constrictive pericarditis (CP) and restrictive cardiomyopathy (RCM). Patient population (retrospectively registered) included those presenting with heart failure due to CP (n = 51), RCM (n = 47), and patients without heart failure symptoms (n = 53). Longitudinal, radial, and circumferential strains/strain rates for left ventricular segments were processed into topological feature vectors using Machine learning PH workflow. In differentiating CP and RCM, the PH workflow model had a ROC AUC of 0.94 (Sensitivity = 92%, Specificity = 81%), compared with the GLS model AUC of 0.69 (Sensitivity = 65%, Specificity = 66%). In differentiating between all three conditions, the PH workflow model had an AUC of 0.83 (Sensitivity = 68%, Specificity = 84%), compared with the GLS model AUC of 0.68 (Sensitivity = 52% and Specificity = 76%). By employing persistent homology to differentiate the “pattern” of cardiac deformations, our machine-learning approach provides reasonable accuracy when evaluating small datasets and aids in understanding and visualizing patterns of cardiac imaging data in clinically challenging disease states.

Enhancing Botnet Detection in Network Security Using Profile Hidden Markov Models

A botnet is a network of compromised computer systems, or bots, remotely controlled by an attacker through bot controllers. This covert network poses a threat through large-scale cyber attacks, including phishing, distributed denial of service (DDoS), data theft, and server crashes. Botnets often camouflage their activity by utilizing common internet protocols, such as HTTP and IRC, making their detection challenging. This paper addresses this threat by proposing a method to identify botnets based on distinctive communication patterns between command and control servers and bots. Recognizable traits in botnet behavior, such as coordinated attacks, heartbeat signals, and periodic command distribution, are analyzed. Probabilistic models, specifically Hidden Markov Models (HMMs) and Profile Hidden Markov Models (PHMMs), are employed to learn and identify these activity patterns in network traffic data. This work utilizes publicly available datasets containing a combination of botnet, normal, and background traffic to train and test these models. The comparative analysis reveals that both HMMs and PHMMs are effective in detecting botnets, with PHMMs exhibiting superior accuracy in botnet detection compared to HMMs.

Inverting a model of neuro-muscular control to estimate descending activation patterns that generate fast reaching movements.

Reaching movements generally show smooth kinematic profiles that are invariant across varying movement speeds even as interaction torques and muscle properties vary non-linearly with speed. How the brain brings about these invariant profiles is an open question.We develop an analytical inverse dynamics method to estimate descending activation patterns directly from observed joint angle trajectories based on a simple model of the stretch reflex, and of muscle and biomechanical dynamics. We estimate descending activation patterns for experimental data from eight different planar two-joint movements performed at two movement times (fast: 400 msec; slow: 800 msec). The temporal structure of descending activation differed qualitatively across speeds, consistent with the idea that the nervous system uses an internal model to generate anticipatory torques during fast movement. This temporal structure also depended on the co-contraction level of antagonistic muscle groups. Comparing estimated muscle activation and descending activation revealed the contribution of the stretch reflex to movement generation that was found to set in after about 20 percent of movement time.

"You Take This Day by Day, Come What May": A Qualitative Study of the Psychosocial Impacts of Living with Duchenne Muscular Dystrophy.

Studies have reported health-related quality-of-life impacts of Duchenne muscular dystrophy (DMD); however, further research is needed to understand how those with DMD experience their condition and how psychosocial impacts evolve over time in response to disease progression. This qualitative study explores the social and emotional implications of key transitions, challenges and adaptations throughout the disease course from the perspective of patients and family caregivers. Semi-structured interviews were conducted with men and boys with DMD, and/or their caregivers, in the USA. Thematic analysis was used to examine patterns in data collected across the interviews. Nineteen participants were included. Three major themes were identified: (1) barriers to participation are multifaceted; (2) an emotional journey shaped by 'inevitable progression;' (3) family provides critical tangible and emotional support. This study illustrates that psychosocial impacts of DMD are shaped by knowledge of the condition's natural history alongside other factors including the extent of social barriers, personal growth and adaptation, and family support. Findings provide insight into the strength and resilience with which individuals and their families respond to daily challenges and major clinical milestones and highlight the relative importance of loss of upper limb function as a transition in DMD affecting health-related quality-of-life.

Exploring Attribute Associations in Pedestrian-Involved Hit-and-Run Crashes through Cluster Correspondence Analysis

Pedestrian-involved hit-and-run (PIHR) crashes represent a significant public health concern, and identifying patterns in these crashes can aid in developing effective countermeasures. Cluster correspondence analysis (CCA) is a multidimensional statistical technique that combines dimension reduction and clustering to identify patterns in categorical data. This method provides insights into underlying patterns and relationships among categories. The current study analyzed a Louisiana crash dataset of 2,201 PIHR crashes from 2015 to 2019 using CCA to identify underlying patterns. CCA identified six clusters, examined the top associative attributes, and assessed their cluster-to-dataset percentage ratio. The first two clusters, representing 66% of PIHR crashes, mainly involved crashes on city streets, occurring primarily during early night (7 to 11 p.m.) in Cluster 1 and the afternoon (12 noon to 4 p.m.) in Cluster 2. Clusters 3 and 4, accounting for 30% of PIHR crashes, predominantly exhibited crashes on U.S. and state highways. Cluster 4, which featured fatalities, primarily concentrated on state highways during the early morning hours (4 to 6 a.m.). Meanwhile, Clusters 5 and 6 focused on high-speed highways, specifically interstates involving pedestrian fatalities. A discussion on implementing strategic countermeasures tailored to the distinct characteristics of each cluster is presented. Alongside improvements in context-based countermeasures to ease pedestrian movement and enhance their visibility, strategies such as advocating for stringent hit-and-run laws, incentivizing the use of dashcams, and broadly publicizing resources for crash reporting are projected to be highly effective in curbing PIHR crashes.

AI-Enhanced evaluation of YouTube content on post-surgical incontinence following pelvic cancer treatment

BackgroundSeveral pelvic area cancers exhibit high incidence rates, and their surgical treatment can result in adverse effects such as urinary and fecal incontinence, significantly impacting patients' quality of life. Post-surgery incontinence is a significant concern, with prevalence rates ranging from 25 to 45% for urinary incontinence and 9–68% for fecal incontinence. Cancer survivors are increasingly turning to YouTube as a platform to connect with others, yet caution is warranted as misinformation is prevalent. ObjectiveThis study aims to evaluate the information quality in YouTube videos about post-surgical incontinence after pelvic area cancer surgery. MethodsA YouTube search for "Incontinence after cancer surgery" yielded 108 videos, which were subsequently analyzed. To evaluate these videos, several quality assessment tools were utilized, including DISCERN, GQS, JAMA, PEMAT, and MQ-VET. Statistical analyses, such as descriptive statistics and intercorrelation tests, were employed to assess various video attributes, including characteristics, popularity, educational value, quality, and reliability. Also, artificial intelligence techniques like PCA, t-SNE, and UMAP were used for data analysis. HeatMap and Hierarchical Clustering Dendrogram techniques validated the Machine Learning results. ResultsThe quality scales presented a high level of correlation one with each other (p < 0.01) and the Artificial Intelligence-based techniques presented clear clustering representations of the dataset samples, which were reinforced by the Heat Map and Hierarchical Clustering Dendrogram. ConclusionsYouTube videos on "Incontinence after Cancer Surgery" present a "High" quality across multiple scales. The use of AI tools, like PCA, t-SNE, and UMAP, is highlighted for clustering large health datasets, improving data visualization, pattern recognition, and complex healthcare analysis.

Predictive modelling of transport decisions and resources optimisation in pre-hospital setting using machine learning techniques.

The global evolution of pre-hospital care systems faces dynamic challenges, particularly in multinational settings. Machine learning (ML) techniques enable the exploration of deeply embedded data patterns for improved patient care and resource optimisation. This study's objective was to accurately predict cases that necessitated transportation versus those that did not, using ML techniques, thereby facilitating efficient resource allocation. ML algorithms were utilised to predict patient transport decisions in a Middle Eastern national pre-hospital emergency medical care provider. A comprehensive dataset comprising 93,712 emergency calls from the 999-call centre was analysed using R programming language. Demographic and clinical variables were incorporated to enhance predictive accuracy. Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and Adaptive Boosting (AdaBoost) algorithms were trained and validated. All the trained algorithm models, particularly XGBoost (Accuracy = 83.1%), correctly predicted patients' transportation decisions. Further, they indicated statistically significant patterns that could be leveraged for targeted resource deployment. Moreover, the specificity rates were high; 97.96% in RF and 95.39% in XGBoost, minimising the incidence of incorrectly identified "Transported" cases (False Positive). The study identified the transformative potential of ML algorithms in enhancing the quality of pre-hospital care in Qatar. The high predictive accuracy of the employed models suggested actionable avenues for day and time-specific resource planning and patient triaging, thereby having potential to contribute to pre-hospital quality, safety, and value improvement. These findings pave the way for more nuanced, data-driven quality improvement interventions with significant implications for future operational strategies.

Parallel Chords: an audio-visual analytics design for parallel coordinates

AbstractOne of the commonly used visualization techniques for multivariate data is the parallel coordinates plot. It provides users with a visual overview of multivariate data and the possibility to interactively explore it. While pattern recognition is a strength of the human visual system, it is also a strength of the auditory system. Inspired by the integration of the visual and auditory perception in everyday life, we introduce an audio-visual analytics design named Parallel Chords combining both visual and auditory displays. Parallel Chords lets users explore multivariate data using both visualization and sonification through the interaction with the axes of a parallel coordinates plot. To illustrate the potential of the design, we present (1) prototypical data patterns where the sonification helps with the identification of correlations, clusters, and outliers, (2) a usage scenario showing the sonification of data from non-adjacent axes, and (3) a controlled experiment on the sensitivity thresholds of participants when distinguishing the strength of correlations. During this controlled experiment, 35 participants used three different display types, the visualization, the sonification, and the combination of these, to identify the strongest out of three correlations. The results show that all three display types enabled the participants to identify the strongest correlation — with visualization resulting in the best sensitivity. The sonification resulted in sensitivities that were independent from the type of displayed correlation, and the combination resulted in increased enjoyability during usage.

Enhanced Diabetic Retinopathy Diagnosis: A Comparative Analysis of models with an Ensemble classifier and Deep Q Learning

- A novel method that combines the strengths of different classifiers such as Naive Bayes, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM) is introduced in this paper. This method tackles the urgent need for cutting-edge diagnostic methods in the field of ophthalmology, mainly for the identification of diabetic retinopathy (DR). The approach is ensemble-based. Classical methods in retinal analysis of images often fail as they are static and are unable to adjust to the unique details that each distinct image presents. This constraint results in less accurate and precise diagnostic results, highlighting the urgent need for more adaptable and dynamic methods. The suggested model differs significantly from previous methods. Through the use of an ensemble approach, it capitalizes on the distinct advantages of each classifier: the MLP process's sophisticated feature extraction skill, Naive Bayes' probabilistic analysis, and SVM's non-linear pattern recognition capacity. By combining these techniques, the inherent drawbacks of utilizing a single strategy are addressed, guaranteeing a more thorough examination of retinal samples and images. The core of this idea is the system using Deep Q Learning (DQL) for adaptive classifier selection. Using learned Q Values for various contexts, this reinforcement learning technique selects the best classifier adaptively for each unique retinal image, hence optimizing the ensemble. This approach not only advances diagnosis accuracy and precision but also guarantees ongoing learning and adaptation to keep up with changing data patterns and advances in imaging technology. Extensive experiments on the IDRiD &amp; EyePACS Dataset show the effectiveness of this model with a 5.5% increase in overall accuracy with other performance metrics, the results show a significant improvement over the current method. They represent a significant advancement in the timely and accurate identification of diabetic retinopathy, which will ultimately benefit patients and lessen the strain on healthcare systems. Thus, this work represents a major step forward in patient care as well as a technological advance, opening the door to more efficient supervision and treatment of retinal illnesses.

Detection and Classification of Wilting in Soybean Crop using Cutting-edge Deep Learning Techniques

Background: This paper employs deep learning in the classification of soybean wilting, a plant health indicator affected by external pressures, using a Convolutional Neural Network (CNN) with a pre-trained model. It highlights the promise of deep learning in agriculture by examining the relevance of wilting, evolution in the agricultural sector and applications in crop wellness monitoring. Methods: A CNN is used in the study to classify soybean withering, with special attention to the VGG16 pre-trained model. Deep learning’s ability to interpret complex data patterns is harnessed for intelligent and accurate wilting detection. A smart detection system tailored for soybean wilting is developed, incorporating recent advancements and addressing associated challenges. Result: The CNN model, notably VGG16, achieves 76% overall accuracy in distinguishing healthy and wilted soybean leaves, signifying a transformative shift in soybean crop health management. The approach offers a precise, efficient and sustainable solution supported by state-of-the-art CNN technology, advancing soybean cultivation practices.

Enhancing Student Performance Prediction Using a Combined SVM-Radial Basis Function Approach

This research aims to improve student performance predictions using a combined SVM (Support Vector Machine) and radial basis function (RBF) approach. The developed model utilizes a combination of the strengths of SVM in handling class separation and the ability of RBF to capture complex patterns in data. Student assessment data, including math, reading, and writing scores, is used as a feature to predict student performance on tests. Preprocessing steps, including feature normalization and label encoding, are applied to prepare the data for model training. Next, the SVM model with the RBF kernel is initialized and optimized using GridSearchCV to find the best parameters. Model evaluation was carried out using the R2 metric to evaluate how well the model predicts student performance. Experimental results show that the combined SVM-RBF approach can improve student performance predictions with fairly accurate prediction results of 88%. The practical implication of this research is the development of a more accurate model for predicting student performance, which can be used as a tool to improve educational interventions and decision-making in educational institutions.

A machine-learning approach to investigating the complexity of theory of mind in individuals with schizophrenia

Individuals with schizophrenia have difficulty attributing mental states to themselves and to others - Theory of Mind (ToM). ToM is a complex, multifaceted theoretical construct comprising first and second order, first and third person, egocentric and allocentric perspective, and cognitive and affective ToM. Most studies addressing ToM deficit in people with schizophrenia consider it an "all-or-nothing" ability and use a classical statistical methodology to test a null hypothesis. With the present study, we investigated ToM in individuals with schizophrenia, considering its complex nature and degrees of impairment. To do this, we used a machine-learning approach to detect patterns in heterogeneous and multivariate data. Our findings highlight the complex nature of ToM deficit in individuals with schizophrenia and reveal the relationship between various different aspects of ToM.

Attention-enhanced conditional-diffusion-based data synthesis for data augmentation in machine fault diagnosis

Data scarcity and class imbalance are pervasive challenges in machine fault diagnosis, impeding the development and broad adaptation of accurate and reliable deep-learning-based fault detection systems. To address these issues, we propose a novel attention-enhanced conditioning-guided diffusion-based approach for synthesizing additional training data. By generating noisy motor-current signals and employing a diffusion process to reduce noise levels, we obtain synthetic data that closely resembles the statistical properties and patterns of real-world data. The quality of the synthesized samples is assessed using a four-layer, pre-trained convolutional neural network, achieving an impressive classification rate of 96.39%. This demonstrates the effectiveness of our attention-enhanced conditioned diffusion-based model in synthesizing critical features for fault detection, making it well-suited for the introduced fault detection task. To further investigate the quality of the synthesized data samples, we implement a specific training strategy that uses the generated samples as training data. The trained diffusion model generates samples per class, which are then used to train a simple four-layer convolutional neural network for fault classification. Remarkably, the model achieves an accuracy of 98.96% when evaluated with real inverter signals, indicating that the generated samples effectively capture the distribution of real data. For further evaluation, multiple scenarios are created confirming the value of the additionally synthesized samples for real-world bearing fault detection. Our approach presents a well-suited solution to the data scarcity and class imbalance problem in machine fault diagnosis. The proposed method enhances the accuracy and reliability of fault diagnosis systems, offering valuable insights for various real-world condition monitoring applications.

Fraud Detection and Prevention in E-commerce using Decision Tree Algorithm

Abstract: Fraud detection is an important part of e-commerce because it helps prevent fraud such as illegal transactions, identity theft, and money laundering. Recently, there has been a lot of literature on the application of machine learning algorithms to identify e-commerce fraud. These algorithms work by learning patterns in data that indicate fraud. Pattern checking deals with discovering differences in data, such as unusual products, locations, or behavior outside the norm for certain users, through machine learning. In this project, we propose a decision tree algorithm to detect fraud in e-commerce using newly generated data from various online products on e-commerce sites. In addition to fraud detection, we also provide advice on fraud prevention. We propose a new security model that will prove the user's identity. In this security model, users are required to register their profile with some questions. Our security systems will display relevant images in response to the registration question. The user has to click on the correct answer image within the time limit. We will ask the user 3 questions in graphic format. If the user selects the correct answer, the user will be considered a real user.

Data patterns that reliably precede US recessions

AbstractThis paper proposes a method of forecasting US recessions beginning with data displays that contain the last 12 quarters of seven US expansions. These end‐of‐expansion displays allow observers to see for themselves what is different about the last year before recessions compared with the two earlier years. Using a statistical model that treats this historical data as draws from a 12‐dimensional multivariate normal distribution, the most recent data are probabilistically inserted into these images where the recent data are most like the historical data. This is a recession forecast based not on presumed patterns but on patterns revealed by the data.

Real-Time Violence Detection in Surveillance Videos Using Deep Learning Approach

Abstract: In order to tackle the ongoing problem of keeping fruits fresh, this research presents an automated freshness detection. The ever-growing presence of surveillance cameras has undoubtedly enhanced public safety. However, manually reviewing vast quantities of video footage for violence detection remains a tedious and error-prone task. This paper proposes a deep learningbased approach for anomaly detection in surveillance videos, with a specific focus on identifying violent activities. Traditional surveillance methods often lack the sophistication to distinguish between normal behaviour and potentially threatening actions. Deep learning offers a significant advantage by automating the identification of deviations from established behavioural patterns in video data. This automation enables real-time analysis of footage, potentially signifying the occurrence of violent incidents and allowing for a swifter response from security personnel. This research delves into the application of deep learning models for violence detection in surveillance footage. We begin by discussing the limitations inherent in conventional methods, such as motion detection and manual review, which struggle to capture the nuances of human behaviour. Subsequently, we explore the advantages that deep learning-based approaches bring to the table, including their ability to learn complex patterns from large datasets and identify subtle anomalies that might escape human observation.

Enhancing Solar Forecasting Accuracy with Sequential Deep Artificial Neural Network and Hybrid Random Forest and Gradient Boosting Models across Varied Terrains

AbstractEffective solar energy utilization demands improvements in forecasting due to the unpredictable nature of solar irradiance (SI). This study introduces and rigorously tests two innovative forecasting models across different locations: the Sequential Deep Artificial Neural Network (SDANN) and the Deep Hybrid Random Forest Gradient Boosting (RFGB). SDANN, leveraging deep learning, aims to identify complex patterns in weather data, while RFGB, combining Random Forest and Gradient Boosting, proves more effective by offering a superior balance of efficiency and accuracy. The research highlights the SDANN model's deep learning capabilities along with the RFGB model's unique blend and their comparative success over existing models such as eXtreme Gradient Boosting (XGBOOST), Categorical Boosting (CatBOOST), Gated Recurrent Unit (GRU), and a K‐Nearest Neighbors (KNN) and XGBOOST hybrid. With the lowest Mean Squared Error (147.22), Mean Absolute Error (8.77), and a high R2 value (0.80) in a studied region, RFGB stands out. Additionally, detailed ablation studies on meteorological feature impacts on model performance further enhance accuracy and adaptability. By integrating cutting‐edge AI in SI forecasting, this research not only advances the field but also sets the stage for future renewable energy strategies and global policy‐making.